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Abstract 

Digital certificates are used to secure international com- 
putation and data storage grids used for e-Science 
projects, like the Worldwide Large Hadron Collider 
Computing Grid. The International Grid Trust Feder- 
ation has defined the Grid Certificate Profile: a set of 
guidelines for digital certificates used for grid authenti- 
cation. We have designed and implemented a program 
and related test suites for checking X.509 certificates 
against the certificate profiles and policies relevant for 
use on the Grid. The result is a practical tool that assists 
implementers and users of public key infrastructures to 
reach appropriate trust decisions. 

1 Introduction 

The international e-Science grid community is heavily 
dependent on X.509 [11] digital certificates issued by 
certification authorities (CAs). 

The International Grid Trust Federation, working 
through the Open Grid Forum, has defined the Grid Cer- 
tificate Profile as a set of guidelines for digital certifi- 
cates used for grid authentication^] 

A relying party, i.e. anyone who must trust the pub- 
lic key infrastructure in order to make use of the system, 
may wish to check that a CA is compliant with the Grid 
Certificate Profile, or some other set of requirements. 
This cohort would also include grid system administra- 
tors or security auditors acting on behalf of grid users. 

The process to check that a digital certificate is com- 
pliant is largely manual: the relying party must work 

* Ac cepted for SAC 10 March 22-26, 2 010, Sierre, Switzerland. 
1 See |http://igtf.net/| and [http://ogf.org/| 



their way through the Grid Certificate Profile section- 
by-setion checking that the certificate under examina- 
tion meets the requirements of each provision. This 
would typically require the certificate to be output in 
a human-readable form and also in a representation of 
the underlying ASN.l structure. 

As the certificates are computer-readable and the 
Grid Certificate Profile is relatively well-defined, there 
is clearly scope for automation of this compliance 
check. This should make it easier and more accurate 
to check that a CA is complying with requirements, and 
this should allow relying parties to check compliance 
as necessary, rather than relying on informal assertions 
made by others. It should also allow CAs implement- 
ing PKI for the grid to check that they are compliant 
before applying for accreditation as grid CAs, thereby 
reducing effort. 

This is particularly important as the number of grid 
CAs continues to grow as new communities join. In par- 
allel, use of the grid - and with it the importance of the 
trust infrastructure - is increasing. For these reasons, 
we believe that a tool to automate certificate checking 
is important for scalability. 

2 Background 

The European Policy Management Authority for Grid 
Authentication in e-Science (EU Grid PMA) was es- 
tablished in 2004 to develop practices, requirements 
and policies for CA coodination in Europe and interna- 
tionally, emerging from work done in earlier European 
grid projects to create a large international X.509 pub- 
lic key infrastructure for grid authentication [1|. The 

2 See|http://www.eugridpma.org/| 
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Americas Grid Policy Management Authority and the 
Asia-Pacific Grid Policy Management Authority were 
established on a similar basis to coordinate grid authen- 
tication policies in those regions^] and the International 
Grid Trust Federation (IGTF) was established in Octo- 
ber 2005 to coordinate policies and practices between 
the regional grid authentication policy management au- 
thorities ifTUl . The Open Grid Forum has a C A Opera- 
tions working group that works closely with the IGTF 
to establish policies and procedures. 

The conventional approach to building a large-scale 
X.509 PKI is to set up a hierarchy of certification au- 
thorities with a single root CA and a number of sub- 
CAs and sub-sub-CAs with various assurance levels, 
purposes, catchment areas, etc. For technical and orga- 
nizational reasons, the model adopted for grid authen- 
tication can be characterised as a policy-based bridge 
where each CA meeting a policy is accredited by a 
PMA and a relying party who trusts the PMA can trust 
the accredited CAs. 

The grid PMAs have developed Authentication Pro- 
files for different classes of certification authority which 
cover many aspects of grid CA operations. 

One important aspect of CA policy is the certificate 
profile used when issuing a certificate. This defines 
technical requirements that are fundamental to trust. 
The IETF PKI working group (PKIX) is developing a 
standard profile for X.509 certificates for use on the In- 
ternet in RFC 5280 @. Building on this, the IGTF and 
the OGF have developed a Grid Certificate Profile that 
deals with aspects of the profile that are of particular im- 
portance in the context of grid authentication and trust 
0. 

The idea of automated checking of grid certificates 
and profiles has been a topic of discussion in EU Grid 
PMA and its predecessor, the European DataGrid CA 
coordination group, since 2001. Coghlan presented a 
system for checking CA certificate policies and certifi- 
cation practice statements against requirements in 2002 
151 . O'Callaghan and Coghlan developed a system in 
2006 for evaluating trust in Grid CAs which uses infor- 
mation both from CA policies and certificates lTP3l . 

The NIST Computer Security Division have docu- 
mented a suite of tests for certificate compliance with 
RFC 3280 (now obsoleted by RFC 5280, mentioned 
above^] 



tively 
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It seems possible that other PKI implementers have 
taken a similar approach to check that a certificate meets 
specified requirements or is compatible with a given 
profile. However, these tools do not appear to be gener- 
ally available (on the web) or to be documented in the 
literature. 



3 Design 

The main aim was to design a practical tool for use in 
the e-Science grid context. The target users are CA 
operators, CA auditors, and system administrators who 
wish to check for themselves that a CA is operating ac- 
cording to their policy. 



3.1 Languages and Libraries 

To implement the tool a number of programming lan- 
guages and their associated libraries were considered. 
The initial candidates were Kawa Scheme, Java and 
Perl. 

Kawa is an implementation of Scheme on the Java 
Virtual Machine Q. Kawa was considered as earlier 
trust evaluation software was implemented in this envi- 
ronment IfTUl . Kawa can use Java libraries and the ear- 
lier software used BouncyCastle for functions related to 
cryptography and certificates |12|. This candidate suf- 
fers from the relative obscurity and unfamiliarity of the 
language among the target users. 

Java was considered as it is widely available and, 
again, has access to BouncyCastle as a cryptography 
and certificate library. However, despite its widespread 
use it was not considered suitable as it is not a scripting 
language. 

Perl and Python were also considered as scripting 
languages widely used by system administrators. In 
IGTF discussions, Perl was favoured as it met the per- 
ceived technical requirements of the target user commu- 
nity. 

To work alongside Perl, the main candidate cryptog- 
raphy and certificate library was OpenSSL. The library 
is used by major Grid middleware, such as Globus f8j. 
The OpenSSL command-line tool is often used for PKI 
administration tasks; and many accredited grid CA ser- 
vices use OpenSSL at their core. For these reasons 
OpenSSL was chosen. 



4 See [http://csrc. nist.gov/groups/ST/crypto_apps_infra/pki/pkitesting. html] 
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There are a number of available Perl bindings for ac- 
cess to the OpenSSL library and/or command: the can- 
didate modules considered were OpenCA::X509 and 
Crypt::OpenSSL::X5OS0 

The OpenCA::X509 module was developed in the 
OpenCA PKI projecj^] and was considered because a 
number of grid CAs successfully use this software. 
It provides access to OpenSSL through the OpenSSL 
command-line tool (that is, API calls result in spawned 
OpenSSL command processes). This has the disadvan- 
tage of making the software dependent on the human- 
readable output of the command-line. The module ap- 
pears to be dependent on other modules and libraries 
distributed with the OpenCA package, which would 
tend to make it impractical for use as a stand-alone tool 
for systems administrators. 

The Crypt::OpenSSL::X509 module provides access 
to some of the certificate-handling ability found in 
OpenSSL. It interfaces directly with the OpenSSL li- 
brary API. In addition, the module is available in soft- 
ware repositories for popular Linux OS distributions 
(e.g. Fedora and Ubuntu) making it readily available 
for the intended users. The disadvantage of the Crypt: :- 
OpenSSL::X509 module is that it does not provide ac- 
cess to X.509 certificate extensions and some other as- 
pects of the OpenSSL API. This module was chosen as 
the basis for certificate handling due to its availability 
and the possibility that improvements to functionality 
(described below) might be distributed by Linux ven- 
dors in future. 

3.2 Test Framework 

The requirements on the design of the tool were to pro- 
vide a way to specify assertions and comparisons of cer- 
tificates against profiles; and to provide a framework to 
evaluate a certificate against a suite of these tests and 
get a pass/fail result, a trust score, or advice on possible 
problems. 

The first step was to design the format for the test 
suites. Ideally, a test format should be portable so that 
different implementations can share and re-use tests. A 
portable test format in XML, S-expressions, or JSOnQ 

5 Other modules which were not considered at the time but which 
may merit investigation are Crypt: :X509 and Convert::X509. Both of 
these modules are available through CPAN jhttp://cpan.org( . 

6 See |http://openca.org/| 

7 See |http://json.org/| 



was considered. The disadvantage was that in each case 
a language would need to be defined for assertions and 
comparisons, and for access to certificate fields and at- 
tributes. 

As Perl had been chosen as the de facto implemen- 
tation language, an alternative approach is to structure 
the tests as fragments of Perl code arranged in an asso- 
ciative array where each key was the name of a test and 
each value was a test function. This design is similar 
to that of O'Callaghan |[T3l . An initial prototype was 
created according to this design but displayed the dis- 
advantage that it did not directly provide a framework 
for collating test results. 

A third alternative is to use the standard Perl test 
framework, which produces output in the Test Anything 
Protocol (TAP) formaj^] Many Perl modules include 
a suite of functional tests written for the Test::Harness 
module using the Test::More module. Test::More pro- 
vides a set of commands for assertions and comparisons 
between expected and computed values: is, isnt, like, 
unlike, etc. Test::Harness allows suites of tests to be run 
and the results collected and presentecj^] The test format 
provided by Test::More is very much tied to Perl, but is 
eminently practical given the requirements. 



3.3 Test Suites 

The test suites for the Grid Certificate Profile are de- 
signed to follow the structure of the document as far 
as possible, with a test for each testable provision. The 
Profile contains provisions related to different classes of 
certificates: those belonging to CAs and those belong- 
ing to end entities, i.e. hosts, persons and robot^] It 
is logical to provide test suites for each class. In addi- 
tion, the patterns used for test suites should be applica- 
ble to other tests, such as security vulnerability checks 
or checks on certificate revocation lists (CRLs). 



8 See |http://testanything.org/| 

5 Test::Harness was used rather than the newer TAP::Harness be- 
cause a version of the former is typically distributed with Perl. 

'"where a robot is an automated client entity that is not a natural 
person. 
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4 Implementation 

The implementation can be divided into three parts: the 
checkcerts.pl script, which runs the tests and collects 
the results; the test suites, specifically the test suites for 
the Grid Certificate Profile; and the enhancements made 
to the Crypt::OpenSSL::X509 library to allow imple- 
mentation of the necessary tests. 

4.1 Test Script 

The central script is a simple wrapper around the Test::- 
Harness module. It takes as parameters a list of test 
files and a list of certificate files (which should be in 
PEM format). The aggregate option causes all tests on 
all certificates to be executed in a single test run to give 
an overall PASS or FAIL result, which may be useful to 
verify compliance of a set of certificates. 

As the Test::Harness module does not provide a 
mechanism for passing parameters (such as the paths 
of certificate files) to an individual test or test suite it 
was necessary to work around this limitation. 

The approach taken was to output the list of cer- 
tificates to a known file. Each test suite intended 
for use with checkcerts.pl must use the supplied 
CheckCertsTest module, which will open the known 
file and store the list of certificates in an array variable 
available to the test suite. 

4.2 Test Suites 

A typical Perl test suite contains a series of test com- 
mands. To allow multiple certificates to be tested, the 
certificate test suites are structured as a loop over an ar- 
ray of certificates. The first test in the test suite loads 
the certificate to be examined from a file; this can be 
considered a side-effect. 

As the motivating example, the initial effort was di- 
rected towards implementing the provisions of the Grid 
Certificate Profile as a number of test suties, one for 
each class of certificate: CA certificates, host or server 
certificates, personal certificates and robot certificates. 

Each test consists of two parts: a comparison and a 
message. The comparison will typically contain a cal- 
culated or retrieved value, an operator and an expected 
value. For example: 

cmp_ok ($x509->version, '==', 2, 

'Version value MUST be "2" per X.509v3 
(2.1)'); 



In this example, cmp_ok is the test command: it tests 
the retrieved value on the left $x509-> version (the 
certificate format version) against the required value 2 
on the right using the equality operator ==. The final 
parameter to the test command is the output message; 
in this test suite it is the provision stated in Section 2.1 
of the Grid Certificate Profile. If the certificate were to 
fail this test the message would be printed to indicate 
where the problem lies. 

The test framework also supports regular expression 
comparisons, for example: 

unlike ($x509->sig_alg_name, ' /md5/ i' , 
'Message digest MUST NOT be MD5 in new 
CA certs (2.2) ') ; 

That is, the signature algorithm name must not match 
md5. 

In addition to tests on the certificate object ($x509 in 
the examples) the test suites examine the subject name 
and certificate extensions in more detail. These ar- 
eas in particular required enhancements to the Crypt: :- 
OpenSSL::X509 module. 

The unmodified module provides the subject name 
as a string; whereas the profile requires examination of 
the ASN.l type, encoding and value of elements within 
the name. Functions were added to the X509 object to 
return an ordered list of name elements; to query the ex- 
istence of a given type of element in a name; to search 
for an element by type (by type name or by object iden- 
tifier); and to query the encoding of name elements (e.g. 
ia5String, printableString, etc.). These additions al- 
lowed Sections 2.3 and 3.2 to be implementecp"[ 

The unmodified Crypt::OpenSSL::X509 module also 
does not provide general support for access to certificate 
extensions, but the underlying OpenSSL API provides 
functions to retrieve extensions by index and also to 
convert the object identifier (OID) to a readable name. 
This allowed construction of an associative array relat- 
ing extension names to values. An example is: 

my $exts = $x509->extensions_by_name () ; 

ok ($$exts { ' basicConstraints' } ->critical ( ) , 
' basicConstraints SHOULD be marked critical 
(2.4.1)'); 

11 Issuer and Subject names of Certification Authority certificates 
and Subject distinguished names of End-entity certificates respec- 
tively 
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To implement the provisions of Sections 2.4 and 3.3 
of the Grid Certificate Profile (dealing with extensions 
in CA certificates and end-entity certificates respec- 
tively) it was necessary to extend Crypt: :OpenSSL:: - 
X509 to give it some semantic information for rele- 
vant extensions, such as basicConstraints, keyUsage, 
and extendedKey Usage. 

It should be noted that the provisions of the Grid 
Certificate Profile make use of RFC 2119 terminol- 
ogy (MUST, MUST NOT, SHOULD, SHOULD NOT, etc.) 

for requirements [3 |. The tests defined with the Perl 
Test modules either PASS or FAIL: there are no in- 
termediate degrees. If a certificate does not imple- 
ment a SHOULD provision it will FAIL that test. This 
is considered acceptable within IGTF as an unimple- 
mented SHOULD provision must be clearly justified to 
the accrediting PMA. For other uses SHOULD provi- 
sions could be placed in a separate test suite. 

In addition to the Grid Certificate Profile tests, test 
suites have been developed to implement security vul- 
nerability checks highlighted by the IGTF Risk Assess- 
ment Team (or RATj 12 | In particular, these check cer- 
tificates for vulnerable RSA parameters, known-weak 
RSA keys due to a recent Debian vulnerability, and the 
vulnerable hash algorithm MD5. Finally, a small IGTF 
RAT test suite against CRLs has been created to check 
for use of the MD5 hash algorithm. This required fur- 
ther development of the Crypt::OpenSSL::X509 mod- 
ule to support access to CRL objects. 

5 Results 

5.1 Grid Certificate Profile 

The Grid Certificate Profile contains 88 distinct provi- 
sions. Of these, 44 pertain to CA certificates and 65 
to end-entity certificates; 21 are common to both. Of 
these 4 pertain only to host certificates while 61 are 
common to all end-entity certificates, for hosts, persons 
and robots. 

Of these 88 provisions, 80 were found to be imple- 
mentable as automatic tests. The remaining 8 are not 
yet implemented in our system for a number of rea- 
sons. 4 require comparing multiple certificates (e.g. to 
check uniqueness of serial numbers or subject names). 
One requires an online check: that a cRLDistribution- 
Point URI refers to a DER-encoded CRL. A further 3 

12 See |http://tagpma.es.net/wiki/bin/view/IGTF-RAT/| 



require a manual check or a value judgement. For ex- 
ample, whether an OCSP service is of production qual- 
ity would require investigation (from Section 3.3.13). 

In total, 61 tests were implemented. There remain 19 
implementable provisions to be completed. 

We have implemented 69 percent coverage of the 
Grid Certificate Profile and 75 percent of its imple- 
mentable provisions. It may be the case that some pro- 
visions should be given more weight than others. 

5.2 Compliance of IGTF CAs 

At the time of writing the latest version of the Grid 
Certificate Profile test suite for CA certificates was run 
against the CA certificates in the IGTF distribution. 
This gives an overall picture of profile compliance. It 
also highlights some short-comings in the test suite. 

Only 22 out of 91 CA certificates fully passed the 
test suite. Two certicates belonging to one CA (now 
superceded) failed on one or more MUST provisions 
(specifically, the use of the MD5 hash algorithm) and 
another 69 failed on one or more SHOULD provisions. 
The most common causes of failure are given in Table 
1. 



Cause 


Certificates failing 


Serial number 


39 


nsCertType 


32 


Subject Name 


22 



Table 1: Common causes of failures. 



By way of explanation, the serial number should not 
be equal to zercj^J the use of the nsCertType exten- 
sion is deprecated and, if used, must be consistent with 
keyUsage; and the subject name should have the correct 
encoding and components. 

5.3 Compliance over time 

The correlation between compliance and the date when 
a CA was accredited by a grid PMA was investigated. 
Date of CA certificate issue, as indicated by the not- 
Before attribute, is used as a proxy variable for date of 
accreditation. In general, the certificate will be issued 
shortly before accreditation. However, a CA may up- 
date its certificate without necessarily updating its cer- 
tificate profile. A plot of the number of Grid Ceritifcate 

13 This requirement is additional to the Grid Certificate Profile. 
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Profile failures against CA certificate notBefore date is 
shown in Figure [T] 

Overall, a linear fit to the data points indicates a very 
slight increase in failures (a decrease in compliance) 
over time. Note that the number of failures ranges from 
zero to six out of a possible 61. A plot of the average 
number of failures per CA appears to show a decrease 
over time. In addition, it appears that the number of 
compliant CAs is increasing over time but this may sim- 
ply be due to the increase in the total number of CAs. 

6 Discussion 

It is encouraging to see that the approach to trust evalu- 
ation of certificates presented here has allowed us to im- 
plement automated tests for a significant proportion of 
the provisions of the Grid Certificate Profile; potentially 
over 90 percent will be implementable. For IGTF rely- 
ing parties it is reassuring to have the technical means 
to verify compliance to such a large degree. 

It would be desirable to improve the test suites to 
clearly indicate if a known provision has not been im- 
plemented so that a relying party may make the nec- 
essary checks. The implementation of the test suite 
also highlighted the fact that individual provisions in the 
Grid Certificate Profile are not unambiguously labelled. 
This may be useful input for a subsequent document re- 
vision. 

The results for compliance amongst IGTF accred- 
ited CAs are disappointing but not entirely unexpected. 
It should be noted again that the failures were largely 
against SHOULD provisions, i.e. strong recommenda- 
tions. A CA may be able to justify its lack of compli- 
ance. Some of the failures can be explained by virtue of 
the age of the CA certificate. For example, to quote 
from Section 2.4.4 on the use of nsCertType and re- 
lated attributes: The ns* attributes are deprecated and 
SHOULD NOT be included in any new CA certificates. 

The correlation between date of issue of the CA cer- 
tificate and its compliance is interesting. The results in- 
dicate that full compliance cannot be assumed for newly 
issued certificates. One possible cause is older CAs is- 
suing a new certificate using a non-compliant profile. A 
correlation between CA software and compliance with 
the test suite might also be revealing. 

An issue that arose during development of the test 
suites was the distinction between test assumptions and 
test provisions. As mentioned, some provisions in the 
Grid Certificate Profile apply specifically to, say, CA 



certificates. The test suite for CA certificates is written 
with the assumption that it will be run against CA cer- 
tificates. If the CA certificate test suite is passed a cer- 
tificate with, for example, basicConstraints attributes 
that are not appropriate for a CA certificate this will be 
indicated as a specific test failure. 

The test suite for the Grid Certificate Profile does 
not address all aspects of grid CA compliance with 
the IGTF accreditation requirements. For example, the 
IGTF Authentication Profile for Classic X.509 CAs £7) 
has requirements for ceritificate validity period and key 
lengths. In addition, it imposes requirements on CA re- 
vocation services. 

The technical certificate profile checks provided by 
the tool described here are a useful complement to au- 
tomated checks on Certification Practice Statement doc- 
uments [4|. One advantage that this software has over, 
for example, the system (based on Kawa Scheme and 
Java) presented in lfl3l is that it is more practical and 
accessible for system administrators. In addition, it has 
better coverage of certificate profile tests. It would also 
appear to be readily usable with higher-level tools that 
can make use of the TAP output from the Perl Test 
framework. 



7 Conclusions 

A number of items of future work emerge from the re- 
sults of this project. There is a clear need to get as 
close to complete coverage of the Grid Certificate Pro- 
file as possible. In addition, complementary test suites 
for other grid and PKI requirements or standards are de- 
sirable. 

With reference to the software, it would be useful to 
have an online system to allow certificates to be up- 
loaded for testing. It might also be possible to use the 
software as an input to an online certificate status or 
certificate validation service. In addition, since the dis- 
tribution of grid CA certs is easily available a web ap- 
plication to present the current test results for all CAs 
would be attractive. This would be an update to the CA 
Trust Matrices, presented in 0. 

Nevertheless, in its current state the tool and test 
suites as described comprise a useful practical tool for 
grid PKI implementers and users, and indeed IGTF re- 
viewers have begun to make use of it when assessing 
CAs for accreditation. 
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Figure 1: Plot of number of Grid Certificate Profile failures against CA certificate notBefore date. 
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